Virtual Objective Structured Clinical Examination Research Articles

Designing a touchless physical examination for a virtual Objective Structured Clinical Examination

Purpose: Given the COVID-19 pandemic, many Objective Structured Clinical Examinations (OSCEs) have been adapted to virtual formats without addressing whether physical examination maneuvers can or should be assessed virtually. In response, we developed a novel touchless physical examination station for a virtual OSCE and gathered validity evidence for its use.  Methods: A touchless physical examination OSCE station was pilot-tested in a virtual OSCE in which Internal Medicine residents were asked to verbalize their approach to the physical examination, interpret images and videos of findings provided upon request, and make a diagnosis. Differences in performance by training year were explored using ANOVA. In addition, data were analyzed based on a modified approach of Bloom's taxonomy of learning: knowledge, understanding, and synthesis. Results: Sixty-seven residents (PGY1-3) participated in the OSCE. Scores on the pilot station were significantly different between training levels (F=3.936, p=0.024, ηp2=0.11). The pilot station-total correlation (STC) was 0.558, and the item-station correlations (ITC) ranged from 0.115 to 0.571, with the most discriminating items being those that assessed higher orders of learning (understanding and synthesis). Conclusion: This touchless physical examination station was feasible, had acceptable psychometric characteristics, and discriminated between residents at different levels of training.

Should Virtual Objective Structured Clinical Examination (OSCE) Teaching Replace or Complement Face-to-Face Teaching in the Post-COVID-19 Educational Environment: An Evaluation of an Innovative National COVID-19 Teaching Programme.

Background The COVID-19 pandemic brought about drastic changes to medical education and examinations, with a shift to online lectures and webinars. Additionally, social restrictions in the United Kingdom (UK) inhibited students' ability to practice for objective structured clinical examination (OSCE) with their peers. Methods The Virtual OSCE buddy scheme (VOBS) provided a means to practice OSCE skills virtually by linking groups of 2-6 final-year medical students with a junior doctor who had recently passed their exams. Sessions were held virtually, tailored to the needs of each group, in a 3-month period prior to examinations. The scheme ran across two examination periods, 2020/21 and 2021/22, including a total of 13 universities throughout the UK. Results In 2020/21, 96% (n=227) of students described improved confidence in OSCE scenarios. Furthermore, 90% (n=213) reported improvement in communication, 89% (n=211) in diagnosis and clinical reasoning and 86% (n=203) in history-taking skills. Examination and procedural skills proved more challenging to practice virtually, with improvement reported by 31% (n=73) and 15% (n=36) of students, respectively. Ninety-three per cent (n=58) of buddies reported improved lesson planning abilities and 90% (n=57) felt more confident in their teaching. In 2021/22, 90% (n=133) of students felt more prepared for their OSCE. In key skills, improvement was reported by 87% (n=128) in communication, 84% (n=124) in diagnosis and clinical reasoning and 83% (n=123) in history-taking. In this cohort, 40% (n=59) reported improvement in examination skills and 24% (n=36) in procedural skills. Ninety per cent (n=83) of buddies reported an improvement in teaching skills, with 93% (n=85) increasing their confidence to teach. Conclusion VOBS demonstrates the benefits to students and teachers of near-peer OSCE teaching. Given the virtual nature, the main drawback is the inability to practice hands-on examination and procedural skills. This scheme provides insight to educators planning virtual teaching programmes in the future. With the evolution of technology, virtual examination and procedure practice may be possible in the near future. VOBS would suggest that currently, virtual OSCE teaching should be used to complement face-to-face teaching.

Feasibility and reliability of the pandemic-adapted online-onsite hybrid graduation OSCE in Japan.

Objective structured clinical examination (OSCE) is widely used to assess medical students' clinical skills. Virtual OSCEs were used in place of in-person OSCEs during the COVID-19 pandemic; however, their reliability is yet to be robustly analyzed. By applying generalizability (G) theory, this study aimed to evaluate the reliability of a hybrid OSCE, which admixed in-person and online methods, and gain insights into improving OSCEs' reliability. During the 2020-2021 hybrid OSCEs, one examinee, one rater, and a vinyl mannequin for physical examination participated onsite, and a standardized simulated patient (SP) for medical interviewing and another rater joined online in one virtual breakout room on an audiovisual conferencing system. G-coefficients and 95% confidence intervals of the borderline score, namely border zone (BZ), under the standard 6-station, 2-rater, and 6-item setting were calculated. G-coefficients of in-person (2017-2019) and hybrid OSCEs (2020-2021) under the standard setting were estimated to be 0.624, 0.770, 0.782, 0.759, and 0.823, respectively. The BZ scores were estimated to be 2.43-3.57, 2.55-3.45, 2.59-3.41, 2.59-3.41, and 2.51-3.49, respectively, in the score range from 1 to 6. Although hybrid OSCEs showed reliability comparable to in-person OSCEs, they need further improvement as a very high-stakes examination. In addition to increasing clinical vignettes, having more proficient online/on-demand raters and/or online SPs for medical interviews could improve the reliability of OSCEs. Reliability can also be ensured through supplementary examination and by increasing the number of online raters for a small number of students within the BZs.

A Telehealth Module and Virtual Objective Structured Clinical Examination of Health Literacy in Pharmacy Education

ObjectiveTo implement an online health literacy (HL) module and evaluate its impact through virtual Objective Structured Clinical Examination (OSCE) on students’ abilities to manage patients with low HL. MethodsStudents virtually engaged in several learning activities related to HL, including practicing using HL assessment tools, designing an information booklet suitable for low HL patients, using readability formulas to lower text readability level to ≤ sixth grade, role-playing scenarios related to HL, and undergoing a virtual OSCE. Student performance on course assessments was tested using Spearman’s rank-order. Students evaluated their OSCE experience in terms of case content, virtual assessment, and logistics; and effectiveness of the HL module and confidence related to HL. ResultsA total of 90 students completed the virtual OSCE; the mean score was 8.8 out of 10 (88%), which correlated with similar course assessments. The average score was 3.46 out of 3.7 for the “gathering information” domain (recognizing risk factors and behaviors, assessing HL and adherence, etc.), and 4.06 out of 4.9 for the “patient management” domain (counseling patients on taking medications, focusing on repeating key messages, offering adherence interventions, etc.). Students responded positively regarding case content and virtual assessment, but less favorably regarding logistics. Feedback on effectiveness of the HL module and confidence in managing patients with low HL was positive. ConclusionAn HL module delivered online was effective in improving student knowledge, abilities, and confidence related to HL. Students achieved high scores in all items in the virtual OSCE, and considered it a measure of communication and clinical skills that is comparable to traditional in-room assessments.

Are we ready yet for digital transformation? Virtual versus on-campus OSCE as assessment tools in pharmacy education. A randomized controlled head-to-head comparative assessment

BackgroundThe global COVID-19 pandemic has influenced pharmacy education including learning, assessment, and exams. In the UAE, pharmacy instructors have adapted several innovative teaching methods to strive for quality learning outcomes. The current trial presented a head-to-head comparative assessment between on-campus versus virtual Objective Structured Clinical Examination (OSCE) with examiners’ and students’ perspectives. AimThe main aim was to compare fourth-year students’ and examiners’ perceptions of the feasibility (time and logistics), stress, performance, and satisfaction between on-campus versus virtual OSCE. MethodA randomized controlled head-to-head comparative assessment between the On-campus and virtual OSCE was conducted to explore performance and satisfaction of pharmacy students and examiners towards the two OSCE settings. The virtual OSCE was carried out directly after the on-campus -OSCE and the setting was designed in a way that aligned with the on-campus OSCE but in a virtual way. Microsoft Teams® breakout room was used as a virtual stations. Respondus-lockdown-browse and Google Meet® were used for proctoring purposes. ResultsStudents who sat for the on-campus assessment were more satisfied with the instructions, the orientation session, the time management, and the overall exam setting, the ability of the exam to assess their communication and clinical skills, professionalism and attitude, and the interactivity of the exam compared to the students who sat for the virtual assessment. Examiners’ perceptions for both settings were the same with the exception of interaction with students (p less than 0.05) as the on-campus OSCE was more interactive. ConclusionStudents still prefer the on-campus OSCE to the virtual OSCE format in many aspects. Nevertheless, virtual OSCE is still a feasible and satisfactory method of assessment when on-campus OSCE is not possible. There is a need of a specialized platform to conduct the virtual OSCE from A to Z rather than maximizing the use of options in the current digital platforms.

Application of OSCE using a Virtual Assessment Platform

Introduction: This study describes the development and implementation of a virtual objective structured clinical examination (OSCE) at the Harvard School of Dental Medicine (HSDM) as an alternative to in-person OSCEs during the COVID-19 pandemic and evaluates its efficacy and reception by students and faculty.Methods: The virtual OSCE was conducted over Zoom (Zoom Video Communications, Inc. San Jose, CA) with students rotating through each discipline via Breakout Rooms. This study examined all 241 OSCE results from the HSDM classes of 2016-2022, with 170 in-person OSCE results and 71 virtual OSCE results. Descriptive and linear regression analysis were used to report the difference in OSCE experience and scores between virtual and in-person OSCEs.Results: Out of 100, the adjusted average score difference was 0.66 higher in virtual OSCE experience compared to in-person assessment, but it was not statistically significant (95%CI=-0.45, 1.78). A total of 35 students (49%) and 14 faculty responded to a survey regarding their virtual OSCE experience. Most students surveyed agreed that the virtual OSCE was effective (94%), accessible (97%), and should be offered post-pandemic (74%). The majority of faculty were neutral regarding the virtual OSCE as a comparable learning experience (76%) and in its accessibility (71%), and only 14% agreed that virtual OSCE should be offered in the future.Conclusion: The results show that the virtual OSCE can be a reasonable alternative to the in-person OSCE when primarily considering its clinical impact and suggest a viable role for virtual OSCE post-pandemic. Technological proficiency training could improve perceptions regarding future virtual OSCE implementation.

Online OSCE in pharmacy education: Strategies to preserve academic integrity of high-stakes assessment

The Objective Structured Clinical Examination (OSCE) is a highly valued performance-based competency assessment that is extensively employed in medical and health professions education. In pharmacy undergraduate programmes, OSCE is an integral component of the curriculum, constituting both formative and summative assessments of the course. When the COVID-19 pandemic posed an overarching challenge in the delivery of face-to-face teaching and learning activities, academic institutions around the world ineluctably transitioned to online mode of education. Conducting OSCEs on virtual platforms presents its unique set of challenges. In the absence of physical isolation and invigilation of students, the risk of cheating and collusion is particularly high during virtual OSCEs. With the experience of conducting high-stakes OSCEs on virtual platforms at two different campuses simultaneously, the authors outline several strategies that can be implemented to ensure the academic integrity of the assessment.

Virtual objective structured clinical examination during the COVID-19 pandemic: An essential addition to dental assessment.

COVID-19 has profoundly affected dental undergraduate teaching and assessment. The pandemic resulted in cessation of face-to face teaching and assessment in many countries, with an associated move online. Objective structured Clinical Examination (OSCE), an important modality for clinical assessment in dentistry and medicine, is not possible with pandemic restrictions in place. As a result, interest in virtual objective structured clinical examination (VOSCE) has been revived. Student and staff evaluation of any assessment process is important, where the views of all involved are required in establishment of authenticity. This papers aims to explore and describe the views of undergraduate dental students and staff in relation to VOSCE MATERIALS AND METHOD: Qualitative methods utilising online focus groups and video recording were used in this study. Five focus groups, involving 24 participants were undertaken. Thematic analysis following a deductive semantic approach was carried out resulting in the identification of six themes relating to the VOSCE: VOSCE preconceptions, examination preparation, examination process, fairness, comparison with OSCE and possible improvements. Consideration of these themes, and their interaction, is likely to prove important for optimisation of this assessment modality. Overall, both staff and students considered the VOSCE a useful and fair examination and a suitable alternative to OSCE. The potential for a number of improvements in the assessment process was identified.

Virtual Objective Structured Clinical Examination Experiences and Performance in Physical Medicine and Rehabilitation Residency.

Virtual education has been described before and during the COVID-19 pandemic. Studies evaluating virtual objective structured clinical examinations with postgraduate learners are lacking. This study (1) evaluated the experiences of all participants in a virtual objective structured clinical examination and (2) assessed the validity and reliability of selected virtual objective structured clinical examination stations for skills in physical medicine and rehabilitation. Convergent mixed-methods design was used. Participants included three physical medicine and rehabilitation residency programs holding a joint virtual objective structured clinical examination. Analysis included descriptive statistics and thematic analysis. Performance of virtual to previous in-person objective structured clinical examination was compared using independent t tests. Survey response rate was 85%. No participants had previous experience with virtual objective structured clinical examination. Participants found the virtual objective structured clinical examination to be acceptable (79.4%), believable (84.4%), and valuable for learning (93.9%). No significant differences between in-person and virtual objective structured clinical examination scores was found for three-fourth stations and improved scores in one fourth. Four themes were identified: (1) virtual objective structured clinical examinations are better for communication stations; (2) significant organization is required to run a virtual objective structured clinical examination; (3) adaptations are required compared with in-person objective structured clinical examinations; and (4) virtual objective structured clinical examinations provide improved accessibility and useful practice for virtual clinical encounters. Utility of virtual objective structured clinical examinations as a component of a program of assessment should be carefully considered and may provide valuable learning opportunities going forward.

A Guide for Medical Students and Residents Preparing for Formative, Summative, and Virtual Objective Structured Clinical Examination (OSCE): Twenty Tips and Pointers.

The most important core competencies for medical learners to master are reviewing history, performing physical examination, communication skills and clinical reasoning. The Objective Structured Clinical Examination (OSCE) provides a consistent, reliable, and valid assessment of these integrated skills and is considered to be the gold standard. OSCEs are advantageous because they provide opportunities in evaluating skills that written tests cannot do (stage 3 of Miller’s Pyramid of Learning). In this article, we have provided tips and helpful pointers to medical students and residents, based on available literature and authors’ expertise in managing formative, summative, and virtual OSCE experiences. In virtual OSCEs, in-person learning objectives need to be modified to the virtual milieu and new competencies such as “webside manner” need to be introduced. Harmonizing the process and content of the OSCEs create operational challenges, thus learning the various moving parts of the OSCEs such as psychometrics, tasks of the standardized patients and checklists will ease optimal performance.

Protecting the Integrity of the Virtual Objective Structured Clinical Examination

Administering objective structured clinical examinations (OSCEs) is a common approach to summative assessment of learner performance in Doctor of Pharmacy (PharmD) curricula.[1][1] The OSCE involves students rotating through multiple standardized stations designed to assess their performance of

Implementation and evaluation of a virtual objective structured clinical examination for pharmacy students

AbstractIntroductionThe University of California, San Francisco (UCSF) School of Pharmacy employs Objective Structured Clinical Examinations (OSCEs) to assess students' skills. The Coronavirus disease 2019 (COVID‐19) pandemic made in‐person OSCEs infeasible. This article describes the implementation and evaluation of a two‐station virtual OSCE (vOSCE) for first‐year pharmacy students.MethodsA summative vOSCE was conducted in June 2020 using Zoom and ExamSoft. Faculty and residents served as standardized patients (SPs) and evaluated student performance. Students and SPs were surveyed on their perceptions post‐vOSCE. A mixed methods analysis was conducted, incorporating thematic analysis of survey responses and quantitative analyses of student performance on the virtual vs in‐person OSCE.ResultsNinety‐six (75%) students and 17 SPs (94.4%) completed the survey. Students and SPs (69% vs 88%, P = .10) felt that vOSCEs were acceptable alternatives for demonstrating knowledge, communication, and telehealth skills. SPs felt that in‐person OSCEs were more effective to assess communication skills and ability to conduct in‐person encounters (P < .05). Qualitative comments from students and SPs corroborated these differences, emphasizing limitations of vOSCEs for demonstrating nonverbal skills, and additional considerations regarding exam administration and assessment. Students who failed an OSCE generally had lower median communication scores (P < .01). Technological difficulties, before OSCE performance, and before telehealth experience were not associated with passing the vOSCE (P > 0.05).DiscussionThe virtual environment did not seem to negatively impact student performance. However, students and SPs felt vOSCEs may be less effective to demonstrate and assess nonverbal communication. To address these limitations, future implementation of vOSCEs may include providing students with instruction on nonverbal telehealth skills and SP training on virtual assessment of nonverbal communication skills.

Telemedicine Training in the COVID Era: Revamping a Routine OSCE to Prepare Medicine Residents for Virtual Care

Background:During the rapid onset of the pandemic, clinicians transitioned from traditional outpatient practice to virtual modalities for providing routine care to patient panels. Like training programs nationwide, telemedicine training and assessment had not been systematically incorporated into our residency. In response, a scheduled Internal Medicine (IM) Objective Structured Clinical Examination (OSCE) was adapted to a remote modality to become virtual care-focused learning experience for trainees and to provide valuable feedback to educators.Methods:Standardized Patients (SPs) rated residents on their communication (including information gathering, relationship development and patient education), patient activation and satisfaction, and telemedicine skills. Analyses included a comparison of domain scores for residents who participated in both the 2020 remote and 2019 in-person OSCEs, and a review of written resident comments about the virtual OSCE.Results:During 2020’s video visit OSCE (VOSCE), residents (n = 23) excelled at nonverbal communication but struggled with virtual physical exams and information gathering. In debrief, residents expressed substantial interest in more opportunity to practice virtual visit skills going forward. In comparing scores of the virtual care (2020) OSCE with the in-person (2019) version, the small subset of residents who participated in both assessments (n = 9) performed similarly on communication skills, patient satisfaction and activation. Patient education scores were significantly lower during the virtual care OSCE (P = .008).Conclusion:Our reformulated OSCE accomplished 3 goals including; (1) physically distancing residents from SPs per COVID regulations, (2) providing residents with the opportunity to practice critical virtual visit skills, and (3) alerting our educators to curricular improvement areas. Our methods are useful for other institutions and have applications to the larger medical education community.

Virtual objective structured clinical examination for family nurse practitioner students using a Zoom platform in the time of COVID-19

Objective: In response to school shut downs amid the COVID-19 pandemic, nurse educators from the University of Southern California implemented a virtual objective structured clinical examination (OSCE) using standardized patients (SPs) to assess family nurse practitioner (FNP) students’ clinical and communication skills as an alternative to the traditional in-person OSCE format. The intent of this paper is to share the nurse educators’ experiences with the transitional process and students’ feedback about their virtual OSCE experiences.Methods: Students (N = 36) enrolled in a childbearing/childrearing clinical course participated in the virtual OSCE using Zoom. The experience included briefing and debriefing sessions. Students were evaluated for their communication and clinical decision making skills based on their assessment of two adolescent patients: one acute with behavioral problems presenting for a checkup and one with headache.Results: All students who participated in the virtual OSCE experience demonstrated appropriate clinical and communication skills. Students perceived the virtual OSCE as a realistic model for telehealth but missed social interaction with faculty and peers and found their inability to conduct physical exam maneuvers challenging. The majority (79.3%) preferred interacting with patients face-to-face.Conclusions: Virtual OSCEs used as low-stakes formative assessments provide FNP students with effective and valuable learning experiences. Transitioning from in-person to virtual OSCEs using Zoom is feasible but requires extensive collaboration between nursing educators and those with access to simulation facilities, such as faculty from schools of medicine. Findings from this experience will serve as a guide for deliberate process improvements for future iterations.

A pilot of a Virtual Objective Structured Clinical Examination in dental education. A response to COVID-19.

The outbreak of COVID-19 necessitated a move to online teaching and assessment. The objective structured clinical examination (OSCE) has been an integral part of dental examinations for several decades. The COVID-19 pandemic stopped face to face examinations around the world. An Online Virtual OSCE (VOSCE) was developed and piloted for dental undergraduate assessment. This initial report outlines the steps required to run an OSCE online. Planning and preparation required in adaptation of an OSCE for the online environment is dreported. The necessity for familiarisation sessions is addressed, and VOSCE logistics described. With careful planning, the VOSCE is a useful assessment method in difficult times. Feedback from staff and students was favourable. Although significant organisation was required, the examination process worked well for both students and examiners. Despite limitations in relation to technical dental procedures, the VOSCE could be a viable alternative to face to face clinical examination.

Twelve tips for conducting a virtual OSCE

Recently many medical schools have faced the challenge of redesigning their existing assessments to run in a virtual format. We ran a virtual Objective Structured Clinical Examination (OSCE) for a group of final year students assessing clinical communication skills, written communication, practical skills, examination skills and professionalism. OSCEs provide the opportunity to test skills that written papers cannot, so it was important to include such a clinical exam in the portfolio of assessments for graduating students. The virtual OSCE ran smoothly and was successful at discriminating between candidates. In this article, we share twelve practical tips from our experience and the small body of literature on how to successfully design and deliver a virtual OSCE. This format provides an opportunity to run similar assessments in the future if remote assessments or assessments of telemedicine skills are required.

Development and use of a Virtual Objective Structured Clinical Examination.

Development and use of a Virtual Objective Structured Clinical Examination.

Avatar-Mediated Home Safety Assessments: Piloting a Virtual Objective Structured Clinical Examination Station

Avatars and virtual worlds offer medical educators new approaches to assess learners' competency in home-safety assessments that are less time-consuming and more flexible than traditional home visits. We sought to evaluate the feasibility and acceptability of implementing an avatar-mediated, 3-dimensional (3-D) home simulation as a virtual objective structured clinical examination station for geriatric medicine fellows. We developed a 3-D home simulation in the virtual world Second Life (Linden Lab, San Francisco, CA) containing 50 safety hazards that could affect the safety of an elderly person at home. Eight geriatric medicine fellows participated in a 16-station objective structured clinical examination, with one station assigned to the 15-minute 3-D virtual world simulation, where the fellow's "home visit" was performed by navigating his or her avatar in the virtual world simulation. The fellows were instructed to find the home safety hazards in the simulated environment and then provide specific written recommendations. Two reviewers independently scored the fellows' written findings against an inventory-based checklist. The geriatric medicine fellows scored a mean of 43% ± SD 9 on the inventory-based checklist. The scoring of the 2 reviewers showed a high interrater reliability (88%). Six of the 8 participants (75%) rated the simulation as "excellent." Four of the 5 women (80%) and none of the 3 men (20%) participating in the virtual objective structured clinical examination needed navigation assistance in the 3-D virtual house. The 3-D, avatar-based, virtual geriatric home safety objective structured clinical examination is a practical and acceptable alternative to the traditional home safety visits in an objective structured clinical examination setting.